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Priest et al. (this issue) present two investi- 
gations from a national collaboration of spe- 
cial education early childhood researchers en- 
gaged in a multi-site, multi-year effort to de- 
velop and validate general growth outcomes 
for children. The first investigation identified 
a set of outcomes that describe the develop- 
ment of children birth to age 8; the second 
study validated the outcomes. The identifica- 
tion of critical growth indicators, valid over 
an 8-year developmental period, could be ex- 
ceedingly useful. If the outcomes are to ex- 
emplify new directions in assessment and be 
adopted as measures of national goals, then 
the identification methods must be able to 
withstand rigorous scrutiny. The research de- 
scribed in Priest et al. is both interesting and 
puzzling. Of concern is whether the research 
is sufficiently meritorious to result in a valid 
measure of growth outcomes suitable for na- 
tional use. This review raises concerns for fur- 
ther consideration. 

The first study describes the process used 
to select the outcomes. The researchers divid- 
ed themselves into age-related teams to ap- 
proach this task. Following a review of skill 
inventories and curricula programs the teams 
selected important developmental milestones 
for the various ages. Subsequently, they came 
together and used a consensus reduction tech- 
nique to narrow the list to a single set of 15 
general growth outcomes that could be effec- 
tive measures across the three age groups. An 
immediate concern is the process used to iden- 
tify the outcomes. The researchers are from a 
single collaborative project and their disci- 
pline-related expertise is similar. Although 


this might lead toward amicable agreement, 
the downside is that it limits their perspectives 
on effective methods for determining growth 
outcomes and in interpreting the clinical sig- 
nificance of specific skills. 

A second and related concern is the criteria 
the group used in narrowing the list of skills. 
By requiring that the critical markers be ap- 
plicable across all age groups, some important 
skills at each age might have been excluded. 
This compromises the content of the out- 
comes. Had the researchers relied on statisti- 
cal procedures to identify the critical skills 
across ages (if they insist on this criteria), the 
outcome list might be quite different and more 
defensible. With the excellent statistical meth- 
ods available to researchers today, it is diffi- 
cult to understand and justify the use of a lim- 
ited set of clinical opinions as a valid method 
for the selection of critical skills. 

A close examination of the specific out- 
comes raises additional concerns. The final 
outcomes are very familiar, not only to early 
intervention practitioners, but to anyone who 
picks up a flier on screening children in a pe- 
diatrician’s waiting room or scans a parenting 
magazine. Given the field’s familiarity with 
these outcomes, one is puzzled as to what is 
new in these outcomes. The outcomes are es- 
sentially the same skills others have been 
measuring for decades. Yet, the authors sug- 
gest there is no valid system presently in place 
to measure child growth in the five areas of 
development. Is it appropriate or innovative 
for the researchers to return to the very sourc- 
es, whose adequacy they had earlier ques- 
tioned, in order to identify the growth out- 
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comes? Further, as originally worded, some of 
the outcomes are likely to have been validated 
in national norming studies. By rewording 
these statements and placing them in the se- 
lected format, validity, from a measurement 
perspective, is lost. One must now question 
whether the outcomes will be sufficient to 
measure children’s competence in a manner 
that will satisfy the public’s outrage over the 
lack of accountability in the field of education. 

The researchers indicate a future step is to 
operationalize the outcome statements. This 
term implies specificity and perhaps these fu- 
ture statements will address the measurement 
concerns. A logical place to search for oper- 
ational descriptions of the outcomes will be 
the very same sources previously reviewed to 
identify the outcomes. It is expected that each 
of the 15 statements will have several subin- 
dicators. This exercise begins to look circular. 
The content is unlikely to change; redundancy 
is expected to continue. Indicators will again 
be borrowed from their original source and 
will be rewritten or adapted to the specifica- 
tions of the researchers. One might reasonably 
question whether these outcomes, or the forth- 
coming operationalized statements, are the 
markers our nation should use for determining 
the preparedness of children to enter school 
ready to learn. 

The researchers suggest that the outcomes 
or operationalized statements will permit them 
to identify instructional targets for individual 
children. The birth to 8-year period of devel- 
opment is the most rapid time of growth. Will 
the statements, even within a 3 -year age span, 
be valid for children whose growth needs are 
likely to be vastly variant? Will these state- 
ments be sufficiently generalizable that facil- 
itators can identify subsequent and appropriate 
targets for instruction even when it means 
moving to the next age range? Concrete, mea- 
surable behavior, such as that required in the 
goal statements on Individualized Family Ser- 
vice Plans or Individualized Education Pro- 
grams, is presently missing from these out- 
comes. Hopefully, the future indicator state- 
ments will include accurate, specific, and 
measurable language so that the progress of 



individuals and of groups of children can be| 
assessed through valid and reliable means. 

Turning to the second investigation, the re- S 
searchers describe a mail survey used to val- 
idate the general growth outcomes. To deter- 
mine whether something is vafid for a nation, ^ 
one must have a valid national sample. From ^ 
the beginning, the investigators buy in to bias: ^ 
They recruit names from membership lists of 
national organizations, which, they acknowl- j 
edge, cannot form a representative sample. | 
The use of a mail survey to validate the out- j 
comes is open to question. According to Ker- j 
linger and Lee (2000), ‘‘the mail questionnaire j 
has serious drawbacks unless it is used in con- | 
junction with other techniques” (p. 603). Con- 1 
tinning their critique of questionnaires, these ^ 
authors indicate that the defects in this meth- 
od, a possible lack of response and the in- 
ability to verify the responses, “. . . are seri- 
ous enough to make the mail questionnaire 
worse than useless . . . .” Kerlinger and Lee i 
provide some concrete targets for use in mea- .■ 
suring return rates: “If mail questionnaires are ^ 
used, every effort should be made to obtain | 
returns of better than 80-^90%, and lacking | 
such returns, to learn something of the char- 
acteristics of the nonrespondents” (p. 603). A j 
response of less than a third of the surveys | 
mailed to the parent group, and the decision | 
to include the pilot sample even after altering | 
the measure, call into question the decision to 
pursue a path plagued with problems. 

Finally, the statistical methods used to de- 
termine the validity of the outcomes must be j 
reviewed. A 3-point scale (critically impor- f 
tant, very important, or somewhat important) | 
was used by the parents and professionals to | 
rate the statements. The pilot study should | 
have identified this as a problem area. The re- 1 
searchers failed to get the desired discrimi- 
nation on this likert-type rating continuum. 
With a scale of five or seven ratings there is 
a greater chance for variance than with a scale j 
of two or three ratings. The limited response j 
variance confounds the variance and is a 
threat to validity. 

Is this where research is in terms of an- 
swering questions concerning validity of out- 
comes? Given the importance of this topic, j 
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researchers are remiss if they do not use meth- 
odologies that have scientific rigor, such as 
criterion-predication validity, construct-iden- 
tification validity, and confirmatory factor 
analysis techniques (Anastasi & Urbina, 
1997). Zigler and Styfco (2000), commenting 
on the National Education Goal 1 and school 
readiness, state emphatically: “The construct 
of school readiness shares a fault with that of 
social competence: Neither has an authorita- 
tive definition and therefore both lack stan- 
dard measurement .... A program must not 
only have a clear, attainable goal, but there 
must be a way to access whether it is being 
realized” (p. 69). Changes in children’s IQ or 
DQ do not have to be, and hopefully will not 
be, the preferred measurement continuum for 
the attainment of growth outcomes, but the 
forms educational researchers do adopt, must 
be valid and defensible. According to Web- 
ster’s dictionary (Guralnik, 1980) the word 
“valid” is from the Latin word, “validus” 
which means strong and powerful. Webster 
defines this word as . . having legal force; 
properly executed and binding under the law 
. . . well-grounded on principles or evidence; 
able to withstand criticism or objection, as an 
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argument; sound” (p. 1568). That is precisely 
what we need in order to provide convincing 
evidence of facts for our nation and our edu- 
cational system. Perhaps this investigation can 
serve best as a pilot step in the rigorous, high- 
ly skilled endeavor that must be undertaken if 
our nation is to have conclusions that can be 
trusted. 
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